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Capacity 


The Capacity of a channel 
is the maximum, over all input distributions Р(т), 
of the mutual information: 
C z maxI(X;Y) 
Px 


The distribution Pj, that achieves the maximum is 
called the optimal input distribution. 


@Shannon's noisy channel coding theorem: 
Reliable (virtually error-free) communication is possible at 
rates up to C 
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What is the capacity? 
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Feedback 


"Feedback? Pah! Who needs feedback? 
Just use a random code!" 
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Recommended homework 


@Noisy channels - Chapters 8, 9, 10 (10.1-10.4 only) 
^ Exercises 9.17 (p155); 10.12 (172); 15.12 (235) 
© and (if you want more practice) 15.11, 15.13, 15.15 


@nvent a channel to pose to your colleagues: 
what's the capacity of this ?' 


The reading associated with the current lectures is Chapters 3, 21 
(especially sec 21.2), and 22 (especially sec 22.1), and 27. 


Other recommended exercises are listed on handout 2. 
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